Similar to our server compilation retry loop, we can avoid killing the worker on failed server communications by adding the same logic. These failings come from network blips and timeouts mainly, and simply pausing should give the worker another chance at processing the build. This will limit "dropped" builds, and — if the admins choose worker registration — unregistered workers.

Example pipeline:

steps:
  - name: sleep
    image: alpine:latest
    commands:
      - sleep 30

  - name: goodbye
    image: alpine:latest
    commands:
      - echo "goodbye"

Testing process (shell):

$ make up
...
...

# [ send two builds to localhost:8080/webhook ]

# wait for sleep step to run for a bit

$ docker kill server

# wait a few seconds or so

$ docker restart server

The above will result in a complete loss of the build if the registration process is not enabled OR an unregistered worker if the registration process is enabled. Either way, this isn't ideal.

Now, with this change, the worker will process the second build so long as the server isn't down for a long time.